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EXECUTIVE SUMMARY 



This report presents the final recommendations 
of the Special Media Records Project for im- 
proved management and preservation of ma- 
chine-readable records in New York State gov- 
ernment agencies. In 1985, the State Archives, in 
cooperation with the Governor's Office of 
Management and Productivity and 19 State 
agencies, initiated the Special Media Records 
Project. The purpose of this two-year project was 
to assess the adequacy of State government poli- 
cies and procedures for the management of com- 
puter-generated, machine-readable records, and 
to develop a program for the long-term preserva- 
tion of selected, valuable machine-readable 



Computers are used to create, process, and 
store an increasing percentage of the State's 
public records. Much of the information that 
State agencies used to store solely on paper, now 
exists in electronic form on magnetic disks, tapes, 
and floppy disks. The use of automated systems 
can bring tremendous benefits to State govern- 
ment because information in automated systems 
is easier to access, manipulate, and disseminate. 
Yet, the use of computers for record keeping also 
creates problems for State government. 
Computers have contributed to an information 
explosion which has inundated some program 
managers with more information than they can 
handle. 

Even though an important part of the public 
record exists in machine-readable form, comput- 
er-generated records were not integrated into 



New York State's centralized records manage- 
ment programs. Comprehensive records man- 
agement programs include investigation into the 
conditions, quantity, and nature of records in 
State agencies, study of records keeping meth- 
ods, provision for transfer of inactive records to 
the State Records Center, authorization for the 
disposal or destruction of records, and preserva- 
tion of records with enduring value at the State 
Archives. Records management programs are 
only partially developed for hard copy (paper 
and microfilm) records, and few programs ad- 
dress the special problems of managing ma- 
chine-readable records. 

The lack of attention to the management, reten- 
tion, and selective preservation of machine-read- 
able records creates numerous problems for data 
processing departments, program staff, and re- 
searchers. Even though technology is available 
to enhance data sharing and make information 
more accessible, the lack of inlormation about 
the content, condition, and location of electronic 
data processing (EDP) records makes access diffi- 
cult. Without systematic review of data retention 
requirements, some machine-readable records 
are maintained longer than necessary to meet 
the information needs of agencies. Others, which 
may have secondary uses or enduring value, are 
erased or destroyed without the review or autho- 
rization required by law.^ The need for better 
management of machine-readable records is es- 
pecially acute because these records are stored 
on media which will deteriorate without active 



^Section 57.05, Subdivision 1 1 of the Arts and Cultural Affairs Law (formerly Sec. 186 ^1 the State Finance Law) requires review by the 
Attorney General and the Comptroller, and approval by the Commissioner of Education pnor to iK^ destruction of any public records. Public 
records are generally defined as any matenal created or received by a government agency regardless of physical form or media. 
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intervention to increase the longevity of the 
^ records. 

The use of computers for record keeping is also 
having a profound impact on the ability of the 
State Archives to preserve State records with en- 
during value. According to a recent report by the 
Committee on the Records of Government, the 
United States is in danger of losing its documen- 
tary heritage because federal, state and local 
governments are shifting rapidly to electronic 
record keeping without making provisions to 
identify, select, and preserve valuable records in 
machine-readable form.2 The situation in New 
York State is similar to the national picture. 
Archivists have developed procedures and tech- 
niques to transfer valuable machine-readable 
records to repositories and to prolong the life 
span of these records.^ However, better manage- 
ment of machine- readable records while they re- 
main in agency custody and increased resources 
for archival programs are essential in order to 
preserve selected, valuable machine-readable 
records. 

It was assumed from the outset — and this re- 
port confirms — that the archival preservation of 
machine-readable records is dependent upon 
the quality of records and information manage- 
ment programs in State agencies. However, long- 
term preservation of selected machine-readable 
records is not the only reason to improve the 
management of EDP information resources. 
Policies and procedures for the orderly retention 
and disposition of machine-readable records are 
essential to ensure that adequate documentation 
of government activities is maintained, to protect 
the legal and fiscal interests of the State and its 
citizens, and to provide for the efficient adminis- 
tration of government programs. Therefore, the 
recommendations in this report address not only 
the essential needs for archival program devel- 
opment, but also include areas where govern- 



ment-wide initiatives are needed and where 
State agencies must assume more responsibility 
for managing their machine-readable informa- 
tion resources. 

When the Special Media Project began, re- 
sponsibility for records management and 
archival programs was divided between the 
Bureau of Records Management in the Office of 
General Services and the State Archives in the 
State Education Department. Fundamental 
changes in the organization and administration 
of records management activities in New York 
State now provide the basis for a more compre- 
hensive approach to the management and selec- 
tive preservation of machine-readable records. 
As of April 21, 1987, statutory responsibility for 
Executive Branch records management pro- 
grams was transferred to the State Education 
Department and combined with the archives pro- 
gram. An expanded records management func- 
tion is being developed in the Education 
Department by the Stale Archives and Records 
Administration (SARA). One goal of this program 
is to integrate machine-readable records issues 
into records analysis, retention scheduling, and 
archival preservation. 

This report addresses the archival and records 
management problems of electronic records in 
the form of a five-year strategic plan. The report is 
divided into three sections. Statewide records 
management and archival programs which ac- 
commodate modern information technologies 
must respond to dynamic changes in the ways 
records are created, maintained, and used. 
Therefore, the first section of the report discusses 
the impact of key trends in information technolo- 
gy on the capacity of State agencies and SARA to 
manage and selectively preserve records to meet 
legal, fiscal, administrative, and research needs. 
The record keeping implications of five important 
developments are analyzed: 1) the increased use 



^ommiUee on the Records of Government, Report (Washington, D.C., March 1985). 

%argaret Hedstrom, Archives 8c Manuscripts. Machine Readable Records (Chicago. Society of American Archivists, 1985). 
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of computers for record keeping, 2) wider distribu- 
tion of information and records, 3) decentraliza- 
tion, 4) increasingly diverse applications, and 5) 
the use of information systems design methodolo- 
gies. This section also examines the relationship 
between computerized and hard copy records 
and the changing organizational context in 
which records are created and managed. 

The second section of the report discusses 
seven bioad objectives for records management 
and archival programs which will help SARA 
and State agencies respond to the challenges 
posed by new record keeping technology. During 
the next five years, SARA will work with agencies 
and with government-wide advisory bodies to 
develop guidelines for the identification, descrip- 
tion, retention and selective preservation of ma- 
chine-readable records. Implementation of such 
programs should ensure that government pre 
grams are adequately documented, and that in- 
formation in machine-readable form is accessible 
(with appropriate safeguards for confidential 
data) to State government and the public. SARA 
vvill also develop programs in education, train- 
ing, technical assistance, and will improve off-site 
storage facilities to help agencies better manage 
EDP records. To meet its responsibility to preserve 
the historically- valuable records of State govern- 
ment, the State Archives will develop its capacity 
to select, accession, describe, preserve and make 
available to researchers machine-recdable 
records with enduring value. 

The final section of the report outlines in detail 
27 activities that State agencies and SARA must 
undertake to accomplish better management of 
machine-readable records during the next five 
years. Each activity and each objective is linked 
to a SARA Bureau in its State Government 
Records Division: Records Analysis and 
Disposition, Agency Services, and Archival 



Services. For each activity a time frame and brief 
explanation are included. 

To accomplish the objectives set out in this 
plan, SARA will utilize existing resources and will 
require additional resources. SARA will request 
funding for two additional archivists, beginning 
in FY 1989-90, one in Records Analysis and 
Disposition and one in the Collections 
Management Unit, Bureau of Archival Services. 
These positions are needed to establish guide- 
lines for retention of EDP records in machine- 
readable form, to evaluate EDP records that hove 
potential archival value, and to acquire and pre- 
serve selected valuable EDP records. SARA will 
request funding for one additional Public Record 
Analyst in FY 1990-91 to develop programs for 
managing records from automated office infor- 
mation systems. The equivalent of approximately 
1 to 1.5 Full Time Equivalents (FTE) of existing staff 
will also be devoted annually to management 
and archival preservation of EDP records. 
Throughout the period covered by this plan, ap- 
proximately $27,000 is needed each year for con- 
tract data processing services and supplies, and 
for bringing experts in information technology to 
Albany to present workshops and seminars. In FY 
1989-90, SARA will request $150,000 for installa- 
tion of adequate tape storage vaults, and $60,000 
to begin a study of optical disk technology for 
archival preservation. Additional resources may 
he needed, beginning in FY 1990-91 to appraise 
and accession valuable data that is in immediate 
danger of loss or destruction, to make possible 
capital improvements to the Records Center tape 
storage vaults, and to start a tape maintenance 
program. Resource requirements for these pro- 
jects cannot be estimated until additional 
assessments ore completed during FY 1988-89 and 
1989-90. 
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INTRODUCTION 



Background. Several recent studies have 
stressed the inadequacy of records management 
in New York State and proposed new initiatives to 
deal with this problem.^ Few agencies have de- 
veloped programs to manage their paper or ma- 
chine-readable records because of inadequate 
training, absence of centralized direction, and in- 
sufficient resources. In most agencies, there is no 
centralized source of information about which 
records exist, where they are located, or how long 
they need to be kept. Records and data in both 
paper and machine- readable formats may be re- 
tained longer than necessary for agency admin- 
istrative purposes or for research use by parties 
outside the agency, while records of long-term 
value may be destroyed prematurely. 

The lack of centralized information about 
records makes it difficult for users to locate 
records and data that they could use for research 
or administrative purposes. Agencies often collect 
and store redundant data and records resulting 
in a duplication of effort and use of expensive 
storage space. Without better records manage- 
ment it is also difficult for archivists to identify 
those records which should be appraised for their 
long-term value. Less than one-third of all gov- 
ernment records have been effectively adminis- 
tered under the existing records management 
provisions, and with few exceptions, machine- 
readable records have been completely neglect- 
ed. As a consequence, the retention, disposition. 



and selective preservation of most government 
records, and especially of EDP records, are ad 
hoc and haphazard activities. 

In 1984, the State Historical Records Advisory 
Board recommended to the Governor that a com- 
prehensive records management program be de- 
veloped for State government records. In recogni- 
tion of the widespread application of new 
information technology for records keeping, the 
Board recommended that special attention be de- 
voted to the management and preservation 
needs of audiO;visual and electronic records. In 
1985, the State Archives, in cooperation with the 
Governor*s Office of Management and 
Productivity, began the Special Media Records 
Project — a two-year study of computer and 
audio-visual records in State agencies. 

The first stage of the Special Medic Records 
Project was an extensive survey of the nature, 
quantity, and condition of special media records 
in 19 selected New York State agencies. The initial 
findings and recommendations of the project 
were reported in a preliminary report, "Computer 
and Audio-Visual Records in New York State 
Government," April, 1986.^ The preliminary report 
recommended that the State Archives should 
begin a systematic program to identify and ap- 
praise machine-readable records for enduring 
value in State agencies, that the Archives and 
agencies work together to establish programs 
and policies for their management, retention, and 



^New York Stale Historical Recoids Avisory Board Toward a Usable Past, Albany, 1934, Governor s Office of Management and Productivity 
and Now York State Education Dspcrtmont, State Archives, Computet uud Audio'Viz-Lil Records in State Government, Preliminary Report of 
the Special Media Records Project, Albany, 1986, and Governor's Office of Management and Productivity, "Consolidating Archival and 
Records Management Function in New York State: A Plan for the Future/ Albany, 1986. 

^Copies of this report are available from tlie Nev/ York State Archives. 
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archival preservation, and that the Archives ob- 
tain the resources and facilities needed to en- 
hance its own ability to preserve these records. 

In 1985 — about the same time that the Special 
Media Records Project began — Governor 
Cuomo issued an Executive Order which re- 
quired agencies to develop records management 
improvement plans and retention schedules for 
all records, including machine-readable records. 
The Governor's Office of Management and 
Productivity is coordinating and providing over- 
sight for this initiative, the Governor's Records 
Management Improvement Project, by sponsor- 
ing records management training, requiring 
agencies to inventory all records, and reviewing 
records manageraent plans from Executive 
Branch agencies. Following an extensive study 
of the State's records management and archival 
programs by the Office of Management and 
Productivity, the Legislature recently consolidat- 
ed these two programs and transferred responsi- 
bility for centralized records management ser- 
vices from the Office of General Services to the 
State Archives and Records Administration 
(SARA), in the State Education Department, This 
administrative change was intended to increase 
the centralized support and coordination needed 
for an ongoing records management program in 
New York State government. 

Scope of the Report. This report presents the 
final recommendations of the Special Media 
Records Project for the improved management 
and selective preservation of computer-generat- 
ed EDP records in New York State agencies-^ The 
report recommends a series of activities to be 
carried out by SARA and by Executive Branch 
agencies during the next five years in order to 
improve the management of records created 
with new information technologies and to en- 
hance the ability of the State Archives to pre- 



serve selected, valuable records in machine- 
readable form. Many of the recommendations in 
this report may also apply to the Legislative and 
Judicial branches of government and to local 
government records, but the problems of elec- 
tronic record keeping in these governmental or- 
ganizations need further study before precise 
recommendations can be developed,^ 

This report stops short of outlining a broad in- 
formation policy for New York State government, 
A general information policy would address 
such issues as procurement of computer, soft- 
ware and telecommunication systems, long- 
range information systems planning, data shar- 
ing, access, privacy protection and security of 
computerized records, and the development of 
systems design, methodologies. Although the 
project staff believes that such policies are essen- 
tial to improved information management in 
New York State government, no single body can 
assume responsibility for all aspects of informa- 
tion management. It is incumbent upon govern- 
ment-wide policy and advisory bodies to devel- 
op a general information policy and to 
coordinate the numerous areas where these is- 
sues overlap. The recommendations in this re- 
port are limited to measures needed to improve 
the management and selective preservation of 
EDP records because these issues reflect the pri- 
mary mission of the State Archives and Records 
Administration, 

This report is divided into three sections. The 
first section of the report outlines strategic issues 
in the management and archival preservation of 
records which result from the increased use of 
new information technologies. The second part 
of the report discusses seven objectives for the 
improved management of machine-readable 
records and for the long-term preservation of se- 
lected valuable records at the State Archives. 



^SARA has prepared a separate report on audio-visual records. 

^SARA's Division of External Programs, Local Records Bureau provdies records retention guidelines and other services to local govern- 
ments. For an initial assessment of EDP records issues in local governments, ^ New York Local Government Records Advisory Council TTie 
Quiet Revolution: Managing New York's Local Government Records in the Information Age (Albany, N. Y., December 1, 1987). 
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The final section of the report describes 27 activi- 
ties to be carried out by the State Archives and 
Records Administration, with assistance from 



Executive Branch agencies and information 
management policy and advisory bodies, during 
the next five years. 
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PART ONE: 



STRATEGIC ISSUES IN THE MANAGEMENT 
AND PRESERVATION OF EDP RECORDS 



New York's records management and archival 
programs at present are directed primarily to- 
ward hard copy (pap6 ; and microfilm) records. 
The electronic age raises new issues, challenges 
and opportunities for improved delivery of infor- 
mation and for better management of the State's 
information resources. The potential for more ef- 
ficient processing of information, improved ac- 
cess, easier retrieval, and cost-effective storage, 
however, will not simply result as a natural out- 
come of the use of new information systems.^ 
Obtaining the potential benefits of the informa- 
tion revolution will be highly dependent on the 
ability of the State to manage its data and 
records. Such an initiative should start with a 
systematic analysis of existing information re- 
sources and an assessment of the n jeds of agen- 
cies to ensure that the data and records they 
need to carry out their missions are available. 

Most legislation regulating the management, 
disposal, and preservation of public records was 
enacted in the precomputer age. Records man- 
agement programs generally have neglected 
the access, retention, and preservation needs of 
machine-readable records. Since computers are 
used to produce, store and retrieve an increasing 
proportion of the State's records, the lack of pro- 
grams to protect, manage, and selectively pre- 
serve this information will become increasingly 
critical. Machine-readable records must be in- 
corporated into the existing laws, regulations, 
policies, and procedures governing the manage- 
ment of public records to ensure that all of the 



State's information resources will r<^main acces- 
sible to public officials and private citizens. This 
section of the report discusses key issues m the 
management and selected preservation of EDP 
records and proposes a strategic approach to 
their improved management during the next five 
years. 

LI KEY TRENDS IN INFORMATION TECH- 
NOLOGY 

The increased use of computers, the imple- 
mentation of office automation systems for text 
processing and electronic mail, and the develop- 
ment of advanced communications networks are 
symbolic of the information revolution. Computer 
technology used Initially to process numeric 
c'ata, can now be coupled with sophisticated 
software and advanced lelecommunications sys- 
tems to process and distribute almost any type of 
information. The use of computers is no longer 
limited to the processing of numbers and small 
amounts of text. Increasingly all forms of infor- 
mation including text, voice communications, 
maps, graphs, and drawings can be handled by 
advanced information technology. The expand- 
ed use of computers, telecommunications, and 
other information technologies is creating a 
growing pool of Information with which program 
managers, records managers, and archivists 
must cope. 

Increasing use of computers in State govern- 
ment. One of the most obvious key trends in in- 
formation technology is tha increasing use of 
computers for all forms of record keeping and 



^In this report an information system is Jefmed as a colleclion <A c<.imputex hardware, skAIv/oto, procedures, and data urganized to t 
plish a special function or set of functions. 
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communications. The numbei of applications 
supported by centralized, mainframe computer 
systems is still increasing in spite of a parallel 
movement toward decentralized, microcomput- 
er-based systems. The Special Media Records 
Project survey identified more than 385 main- 
frame-based production systems in 19 Executive 
Branch agencies. Based on this survey, it is likely 
that more than 1,000 centralized, automated in- 
formation systems are created and maintained 
by State agencies. Preliminary results from an 
inventory of records in 15 agencies indicate that 
data processing is used to process and produce 
more than 20 percent of the paper and microfilm 
records in New York State government 
agencies.^ 

Wider distribution of information in electron- 
ic form. Expanded telecommunications capabili- 
ties are increasing the distribution of information 
in electronic form. In July 1987, New York State in- 
stalled a new telecommunications system which 
allows transfer of voice or data communications 
over one fiber optics network. This new technolo- 
gy may cdso help to resolve some vital manage- 
ment issues for public sector administrators. 
According to a recent Sperry Corporation survey 
of public officials, one of three constant barriers 
to effective management in the public sector is 
the geographic dispersion of operating units. 
Nearly one-third of the respondents reported that 
they frequently use records and information from 
other government agencies, while nearly half 
sometimes rely on external sources of informa- 
tion, Advanced telecommunications networks 
offer the potential to share inforniation between 
different geographic locations of one agency and 
amc ng different agencies. 

Decentralization. The more recent develop- 
ment of microcomputers, word processors, and 
local area networks is rapidly increasing the ex- 



tent to which program managers, rather than 
EDP professionals, create and manage machine- 
readable records. Office automation systems 
supply information for policy analysis, research, 
and evaluation in New York State agencies but 
there has been no systematic study of office au- 
tomation or electronic mail in State government. 
The Sperry Corporation survey indicates how 
widespread office automation systems have be- 
come in public agencies generally. More than 
two-thirds of the 5,200 public sector officials sur- 
veyed use word processing technology, and 
more than 40 percent rely on computers for re- 
port generation. 

It is unlikely that decentralized systems will re- 
place the large production systems supported by 
mainframe computers. In the future, agencies' 
automated information resources will consist of a 
mixture of centralized and decentralized 
databases, files, and documents. However, the 
availability of low cost and user-friendly micro- 
computer and word processing systems has 
greatly accelerated the creation of records in ma- 
chine-readable form. Many of these records are 
created, accessed, and controlled by staff in pro- 
gram units rather than central EDP units. 
Therefore, programs to improve the manage- 
ment of machine-readable records must reach 
beyond central EDP units to users in program of- 
fices. 

More diverse applications and software. The 

increased sophistication of software is another 
key trend in information systems technology 
which has encouraged the expanded use of com- 
puters for records creation, processing, and stor- 
age. Most mcdnframe-based production systems 
rely on database management and other com- 
plex software to process data, manage editing 
and updates, handle on-line queries, and pro- 
duce a variety of reports and other hard copy 



^Daia collected by the Governor's Records Management Improvement Project, a project sponsored by the Office of Management and 
Productivity to inventory all State records and develop records management improvement plans. 

^^Sperry Corporation, Public Sector 1985 Survey Results, p.2. 
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output. The results of the Special Media Project 
survey suggest that nearly three-quarters of the 
State's centralized EDP systems are designed for 
use with in-house software, database manage- 
ment systems, or statistical packages. Most micro- 
computer-based systems rely heavily on software 
packages for data entry, formatting, and report 
generation. 

The use of sophisticated software has distinct 
advantages for data processing and information 
retrieval. In large production systems, database 
management systems reduce data redundancy 
and allow users to combine data in a variety of 
ways. User-friendly software packages for micro- 
computers and word processing systems enable 
individuals with limited training and virtually no 
knowledge of computer concepts or program- 
ming techniques to use computers. 

The increasing use of sophisticated software, 
however, poses problems for data exchange, in- 
formation management, and archival preserva- 
tion of machine-readable records. The hardware 
and software systems that cannot communicate 
with each other create obstacles to the flow of in- 
formation between systems within one agency, 
among agencies, and between agencies and the 
Archives. The use of data file structures which re- 
quire special software complicate the archival 
preservation of EDP records. To preserve soft- 
ware-dependent data and keep it accessible, 
archives hove to maintain the data and the soft- 
ware or rewrite the data into a format that does 
not require special software. Creating data files 
in a software-independent format increases 
preservation costs if such capabilities are not 
part of the system design. Moreover, the use of 
data structures which do not require software 
has increasing limitations as computers are used 
for a much wider variety of applications, such as 
text processing, graphics, mapping, and comput- 
er-assisted design. Software dependency will be- 
come a major obstacle to the transfer, manage- 
ment, and preservation of information in 
electronic form as more sophisticated systems 



are used. 

Use of systems design methodologies. 

Another important trend is the increasmg use of 
more precise design methodologies in the devel- 
opment, implementation, and management of 
automated systems. Studies of data processing 
have demonstrated frequently that information 
systems do not function well unless they serve 
real needs of administrators and program man- 
agers for better processing and retrieval of infor- 
mation. Especially in the development of large 
production systems, a detailed analysis of infor- 
mation needs precedes the development of new 
automated information systems. Such design 
methodologies usually specify the hardware and 
software requirements for the systems, data 
flows, data base management, and the produc- 
tion of output reports. 

Unfortunately, analysis and planning for the 
long-term management of the machine-readable 
and hard copy records produced by automated 
systems rarely is addressed in the system design 
methodology. Decisions about the disposition of 
machine-readable records often are not made 
until a system has reached its capacity in terms 
of disk or tape storage space. Although retention 
cycles usually are established for the input, out- 
put, edit, and other temporary data files, there 
has been little systematic analysis of the reten- 
tion and disposition requirements for master data 
files, databases, and related hard copy docu- 
ments. Better control, management, and preser- 
vation of EDP records require systematic review 
of the data to be created and stored by automat- 
ed systems as part of the design methodology. 

These trends taken together suggest that the 
future creation, storage, distribution and use of 
public records is likely to differ considerably from 
the traditional paper-based information systems 
to which most records managers and archivists 
are accustomed. Accompanying the spread of 
computer technology is a growing diversity of 
applications ranging from large-scale, central- 
ized production systems, to automated text and 
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report generation, to computer-assisted design 
and decision support systems. Increasingly, de- 
centralized and user-driven systems which sup- 
port a wide variety of record keeping 
applications will exist along side of large, cen- 
tralized databases. 

1.2 RELATIONSHIPS BETWEEN PAPER AND 
ELECTRONIC RECORDS 

Overlapping content and functions. Some ex- 
perts predict that the "paperless office" will be 
commonplace by the 1990s as advanced infor- 
mation systems provide instantaneous access to 
information through computers, telecommunica- 
tions, and optical disk systems. To date, however, 
few automated systems hove eliminated the use 
of hard copy documents, and in many cases the 
use of computers has actually accelerated the 
creation of paper records. As a consequence, the 
contents and functions of machine-readable 
records and hard copy documents usually are 
closely related. Data may be extracted from a 
database to produce summary reports, printouts 
may be produced as a database is updated, or 
the correspondence stored on a diskette may also 
exist in an agency's files. For these reasons, data 
and records management must be coordinated 
to account for the closely ^ related content and 
functions of machine-readable records and hard 
copy records. A comprehensive information 
management program must focus first on analy- 
sis of the information in records and then on con- 
trol of the medium on which the information is 
stored. 

Special characteristics of EDP records. EDP 

records differ in a number of ways from the 
paper and microfilm records. In the past, records 
management has focused on managing the 
physical entities on which information is record- 
ed, such as files of paper documents. Information 
in machine-readable form, however can be sep- 
arated easily from its carrier or base and infor- 



mation in machine-readable form can be trans- 
ferred easily from one medium to another. Terms 
such as "original," "duplicate," and "record copy" 
become meaningless because electronic inlor- 
mation can be passed from one medium to an- 
other and duplicated easily. The possibility of 
transferring information in machine-readable 
form from one type of storage medium to another 
also crea ?s a wider variety of options for the 
cost-effective storage and preservation of infor- 
mation. Therefore, an important element of an 
expanded records management program is de- 
termining which storage formats and media are 
most appropriate to meet the legal, fiscal, admin- 
istrative, and research needs for access to infor- 
mation. 

The ease of manipulating information in ma- 
chine-readable form has special advantages for 
users. Researchers who wish to examine a sam- 
ple of cases, study groups or phenomenon with 
special characteristics, or produce statistical 
studies or tabulations can do so much mo^ j easi- 
ly using data m machine-readable form. Tl.e U.S. 
National Archives, the National Archives of 
Canada, and many universities have developed 
data archives programs to meet researchers' re- 
quests for machine-readable records. Not only do 
researchers prefer records in machine-readable 
form for certain types of research, but some im- 
portant historical records are now only created in 
computerized form.^^ 

Machine-readable records will require careful 
handling because of their unique physical 
characteristics. Currently, magnetic tape is the 
most common medium for the long-term reten- 
tion of data in machine -readable form. Unlike 
paper or microfilm, this medium is erasable and 
reusable. Magnetic media are less durable than 
paper or microfilm. Therefore, machine-readable 
records require special storage, careful han- 
dhng, and regular maintenance. Computer 



^ ^For descnption of these programs, see Harold A. Naugler, The Machine-Readable Archives Program at the Public Archives of Canada: 
The Fust Five Years/ and Charles M. Dollar, "Machine-Readoble Records of the Federal Government and the National Archives/ in 
Archivists and Machine-Readable Records, Carolyn Geda, Erik Austin, and Francis Blouin, Jr., eds. (Chicago: Society of American Archivists, 
1980); and Carolyn Geda, "Social Science Data Archives," American Archivist 42 (April 1979), pp. ".58-66. 



hardware and software are essential for retrieval 
of information from EDP records and for interpre- 
tation of their contents. When records are re- 
tained in machine-readable form, records custo- 
dians must ensure that the fragile medium is 
maintained properly and that the means to 
access the information including hardware, soft- 
ware, and adequate documentation are avail- 
able. Without all of these elements — the infor- 
mation and the means to access it — agencies 
will not be able to meet records retention require- 
ments. 

1.3 THE ORGANIZATIONAL CONTEXT 

The organizational context for the creation, 
storage and use of machine-readable records 
also differs from tha: for traditional hard copy- 
documents. With large mainframe-based sys- 
tems, responsibility for the systematic analysis of 
information needs, the design of automated sys- 
tems, the processing and storage of data, and its 
ultimate i-etention or disposal usually is divided 
between users and data processing units. Users 
define the informational content of the records 
and may specify the formats for output reports. 
EDP units usually determine the physical format 
of machine-readable records and control their 
storage. 



With micro-computer based systems, end 
users decide how to organize and access data in 
the system, what documentation (if any) they will 
create to facilitate retrieval of information by out- 
side users, and when they will delete data from 
the system. Unless data management and reten- 
tion guidelines are in place, every user becomes 
an information manager, deciding how to set up 
their electronic filing systems, what information 
to store there, and how long to keep it. Retention 
requirements for machine-readable records 
should be determined by users, but users often 
lack the technical knowledge and familiarity with 
machine-readable records to make sound reten- 
tion recommendations. Therefore, EDP units often 
suggest data retention guidelines and usually 
have primary responsibility for implementing 
them. Because responsibility for the creation, 
storage, management and preservation of ma- 
chine-readable records is shared, users, data 
processing units, agency records managers, and 
SARA staff must develop reasonable ways to di- 
vide these responsibilities and to coordinate in- 
formation management activities. 
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PART TWO: . ' 

OBJECTIVES FOR IMPROVED MANAGEMEN T AND 
PRESERVATION OF EDP RECORDS 



This section of the report discusses seven basic 
objectives for improved management and selec- 
tive archival preservation of EDP records during 
the next five years. These objectives are grouped 
in three general categories v^hich parallel the 
three Bureaus in SARA's State Government 
Records Division: Records Analysis and 
Disposition, Agency Services, and Archival 
Services. Because information management is- 
sues cross many constituencies, key links must be 
established between agency administrators, pro- 
gram managers and other users of EDP systems, 
records managers, agency information special- 
ists, and SARA staff to achieve the objectives dis- 
cussed below. Unlike a model used in some other 
states, no single body in New York State govern- 
ment can or should assume overall responsibility 
for the processing, storage, management and se- 
lected preservation of machine-readable 
records. Instead, the activities of various intera- 
gency groups and agency specialists must be co- 
ordinated at both the policy and operational lev- 
els in order to resolve the key information 
management issues discussed in Part One of this 
report. 

The coordination and cooperation of several 
existing organizations in State government is 
necessary to assess existing records and informa- 
tion management practices and policies for EDP 
records and to recommend changes where need- 
ed. Currently, development and oversight of 
agency records management initiatives is being 
coordinated by the Governor's Office of 
Management and Productivity through the 
Records Management Improvement Project. As 
agencies inventory their records, develop 
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records management plans, and implement 
records management programs, it is essential 
that records in machine-readable form are in- 
cluded. The New York State Forum for 
Information Resource Management, representa- 
tives of the data processing units such as the 
Interagency Committee on EDP, and SARA staff 
should work together closely to ensure that agen- 
cies develop provisions for improved manage- 
ment of EDP records. Closer involvement of EDP 
Directors in the Governor's Records 
Management Improvement Project would pro- 
vide insights into the problems that EDP units 
face in managing the records they process. 

Over the next five years, SARA will work 
closely with the Governor's Office of 
Management and Productivity, the Forum for 
Information Resources Management, and advi- 
sory groups of agency records managers and 
data processing staff to improve records man- 
agement programs in New York State and to en- 
sure that machine-readable records are included 
in these programs. To assist agencies in meeting 
the challenges of new information technology, 
SARA will promote the development of guide- 
lines and procedures for management of ma- 
chine-readable records, provide training and 
technical assistance, assist in evaluating new 
record keeping technologies, and recommend 
improvements in centralized storage facilities 
where needed. These efforts vTill build on the 
work of the Governor's Records Management 
Improvement Project. SARA vdll also develop its 
capacity to identify, select, accession, and pre- 
serve historically valuable EDP records at the 
State Archives. 
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Within agencies, greater effort must be made 
to improve records management and additional 
resources must be devoted to records manage- 
ment activities for all formats of information. 
SARA can provide coordination, training and 
technical assistance, but these centralized ser- 
vices will be effective only if they are comple- 
mented by increased involvement and support 
from agencies for records management activities. 
In the area of machine-readable records, EDP 
units must ploy a key role in helping agency pro- 
gram offices and records managers to develop 
retention guidelines and schedules. Furthermore, 
EDP Directors must be committed to continuing 
involvement in agency records and information 
management programs in order to ensure that 
retention guidelines are implemented as speci- 
fied. 

2.1 RECORDS ANALYSIS AND DISPOSITION 

Improvements in the management of EDP 
records will require increased centralized ser- 
vices for records management to assist agencies 
with analysis of their records. In the area of 
records analysis and disposition, SARA will de- 
velop guidelines for the retention and disposition 
of EDP records, integrate these guidelines into re- 
vised records disposition authorization proce- 
dures, assist agencies in the development of ap- 
propriate retention periods of EDP records, and 
appraise potentially archival records in ma- 
chine-readable form. Overall responsibility for 
implementing EDP records management pro- 
grams will remain with State agencies. 
Therefore, many of the objectives discussed in 
this portion of the report have organizational, 
management, and resource implications for 
agencies. This report assumes that SARA will be 
unable to meet many of its records management 
and archival responsibilities without improve- 



ments in agency records management pro- 
grams. 

OBIECTIVE 1: Develop guidelines and assist 
Executive Branch agencies to implement pro- 
grams which identify, describe, and schedule 
the disposition of data from automated informa- 
tion systems. 

Traditional records management concepts 
and procedures provide a useful starting point 
for the identification, description, and retention 
scheduling of EDP records. As was discussed in 
Part One, information recorded in digitized form 
can be produced on paper or microfilm, and 
hard copy documents overlap in content with the 
information in data files and data bases. 
Therefore, the management of EDP records 
should be coordinated with management and re- 
tention of hard copy documents. Policies and pro- 
cedures for improved records management must 
include both EDP records and hard copy docu- 
ments in order to assist users, records managers, 
EDP specialists, and archivists to determine the 

most appropriate format(s) for retention of infor- 
mation. ^2 

Guidelines for the identification, description, 
and scheduling of data from automated systems 
must be developed to provide the framework for 
integrating machine-readable records into the 
records disposition audit and review process. 
Such guidelines should help agencies to identify 
EDP records that can be disposed after a relative- 
ly short retention period, records that are vital, 
and information systems that are likely to pro- 
duce records with long-term legal, research or 
historical value. In order to ovoid undue burden 
on EDP units which may be required to supply 
descriptive and technical information about ma- 
chine-readable records, it is also important that 
guidelines for the management of data in EDP 



copy docuixidnts, for example, inay be required if they contain signatures or other evidence of legal certifications. In other cases, re» 
tention of information on magnetic tape or some other machine-readable medium may provide a cost effective alternative to the retention of 
large volumes of hard copy documents. 
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systems use an approach that is acceptable to 
the EDP systems community. SARA staff will 
need advice from representatives of EDP units so 
that existing EDP systems management tools can 
be applied wherever they are appropriate. 
Section Three of this report outlines several spe- 
cific activities which will lead to the gradual inte- 
gration of EDP records into broader records man- 
agement programs during the next five years. 

An expanded records management program 
will rely on agencies to inventory, describe, and 
schedule their EDP records in conjunction with 
records managf*>ment initiatives. In submitting 
records retention schedules and disposition re- 
quests, agency records management staffs will 
be expected to coordinate management of EDP 
and hard copy records. SARA archivists and 
records analysts must also be prepared to review 
comprehensive records schedules which include 
records in several formats and to provide advice 
to agencies on the retention of machine-readable 
records. 

The reviewing bodies for disposition requests 
should insist that schedules submitted by agen- 
cies cover agency records in all physical formats, 
taking into account the need for immediate space 
savings through the elimination of unneeded 
paper records. Representatives of the 
Department of Law and the Office of the State 
Comptroller, who review disposition requests, 
should also consider the effects of electronic 
records keeping on the legal and fiscal interests 
of the State and its citizens. Basic policy issues, 
such as the admissability of machine-readable 
records as evidence in courts of law, need to be 
addressed by Department of Law staff with ex- 
pertise in this area. Similarly, guidelines from the 
Office of the State Comptroller regarding the fis- 
cal and audit considerations of financial records 
maintained in machine-readable form require 
clarification. 

OBJECTIVE 2: Ensure that State government 
programs and operations are adequately docu- 



mented for legaL fiscaL administrative^ and 
historical purposes* 

With the increased use of computers, a signifi- 
cant portion of agency information resources 
exist in machine-readable form. These machine- 
readable records contain vital documentation of 
government programs and operations. Sound in- 
formation and records management practices 
must be applied to machine-readable records to 
ensure that adequate documentation of State 
government operations is available to meet 
legal, fiscal, administrative, and historical needs. 

In the past, analysis of the need to keep or de- 
stroy government records generally has not oc- 
curred until a large volume ci records has accu- 
mulated or until an agency requested permission 
to destroy records. This approach to records 
management is unworkable for EDP records and 
is increasingly impractical for managing paper 
records. Records and information managers 
agree that retention periods for machine-read- 
able records should be established as early as 
possible in the record's life cycle so that records 
can be managed from the point at which they 
are created until they are destroyed or trans- 
ferred to the State Archives for preservation. This 
is especially important for EDP records because 
of their fragile storage media and the need for 
hardware and software to access the data. 

During the first two years of an expanded 
records management program, SARA staff in the 
Records Analysis and Disposition Bureau wiU re- 
assess the timing of records analysis and ap- 
praisal studies. Agencies will be encouraged to 
develop records retention schedules for new in- 
formation systems as they are developed and for 
existing records as early as possible in their life 
cycle. These retention schedules will provide 
basic information needed by the agencies that 
review disposition requests to ensure that gov- 
ernment operations are documented adequately 
and to improve the economy and efficiency of the 
information storage and retrieval. Retention 
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schedules may also form the basis of a central in- 
ventory of data which is needed to enhance data 
sharing and reduce data redundancy among 
agencies. The Records Analysis and Disposition 
Bureau will also undertake a major investigation 
of the impact of office automation on record keep- 
ing and recommend appropriate policies and 
guidelines for managing records from decentral- 
ized systems where needed. 

2.2 AGENCY SERVICES 

In order to meet their responsibilities for better 
management of EDP records, agency staff need 
training, technical assistance, and adequate off- 
site storage facilities for magnetic media. In the 
area of agency services, SARA will provide 
records management training and technical as- 
sistance to agency records managers and EDP 
specialists, evaluate the role of the State Records 
Center and other facilities in providing off-site 
storage for machine-readable records, and moni- 
tor changes in record keeping technology. 

OBJECTIVE 3: Increase awareness — among 
records managers* program managers, EDP 
specialists, and users — of the primary and sec- 
ondary values of data# and of policies regarding 
data access* retention* and disposition, and 
long-term preservation. 

One element of an expanded records man- 
agement program will be training and technical 
assistance for agency staff in records manage- 
ment. EDP professionals generally are unfamiliar 
with the records management and archival tech* 
niques used to determine the appropriate reten- 
tion period for records and to develop retention 
and disposition schedules. Therefore, training 
programs should address the special needs of 
users and EDP specialists by teaching them how 
to integrate records retention scheduling into sys- 
tems design methodologies and how to identify 
machine-readable records which are likely to 
hove long-term value for internal agency admin- 
istration or for outside research use. 

Records officers and program managers will 
also need training in the identification, descrip- 
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tion and scheduling of EDP records. Even though 
EDP systems specialists will provide some of the 
descriptive and technical information about elec- 
tronic records, records managers and program 
managers will have to analyze how these 
records are used in their agencies. Records offi- 
cers will also provide a key coordinating role in 
linking EDP records to related hard copy docu- 
ments. Moreover, with the increased use of office 
automation and other decentralized technolo- 
gies, program officers and records managers will 
ha /e to assume greater responsibility for manag- 
ing records produced in automated information 
systems regardless of their physical format. 

OBJECTIVE 4: Enhance the ability of State 
agencies and SARA to manage and, where ap- 
propriate, to preserve records created with mod- 
ern information technology. 

With frequent changes in computers and in- 
formation technologies, there is also a need for 
ongoing technical evaluation and assistance to 
help agencies evaluate the records management 
and archival implications of new technology. 
Currently, for example, agencies need advice 
and assistance with the information manage- 
ment aspects of office automation and optical 
disk technology. The i:^pact of stand-alone and 
network systems on the creation, storage and re- 
tention of basic office communications needs to 
be further evaluated. Records and information 
management guidelines for the electronic and 
hard copy products of office automation systems 
should be developed to help agencies ensure 
that their policy formation and decision-making 
processes are adequately documented. 

Several agencies are investigating the use of 
optical disks as an alternative to hard copy or mi- 
crofilm storage. Feasibility studies investigating 
the use of this technology must address not only 
the immediate cost implications of the use of opti- 
cal disks, but also the long-term records manage- 
ment and archival implications. In order to keep 
pace with such changes in information technolo- 
gy, SARA will monitor developments in informa- 
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tion technology and storage media and provide 
advice and technical assistance to agencies con- 
templating the use of new storage techniques. 

OBJECTIVE 5: Evaluate the role of the State 
Records Center in data storage* security* and 
preservation* and provide improved services 
v'here indicated. 

The Records Center provides off-site storage 
for back-up files and inactive magnetic tapes. 
During the first two years covered by this plan, 
SARA should assess the adequacy of Records 
Center tape storage facilities and services. 
Because magnetic tape is a fragile medium 
which must be stored in areas with stringent tem- 
perature and humidity controls, conditions in the 
Records Center tape storage vaults should be 
evaluated. Security systems should also be eval- 
uated to ensure that vital data is secure and that 
the possibility of unauthorized access to restricted 
data is minimized. 

Even under optimum storage conditions, mag- 
netic tape deteriorates after a period of 2 to 10 
years. Regular maintenance procedures, includ- 
ing rewinding of inactive topes, checking for tape 
deterioration, and the periodic transfer of data to 
new tape, are essential to ensure the readability 
of data when its retention exceeds five years. 
Currently, no centralized service is available to 
provide regular maintenance for magnetic tapes 
or to monitor their readability over extended re- 
tention periods. There may be a role for expand- 
ed Records Center services to provide such 
maintenance services for inactive machine-read- 
able records. 

SARA should assess the need for centralized 
maintenance and conservation services for EDP 
records at the Records Center. Improvements 
may be needed to ensure that temperature and 
humidity in tape vaults meet recommended stan- 
dards for long-term retention of magnetic tape. 
Procedures should also be developed to ensure 
that magnetic tapes stored at the Records Center 
for purposes other than short-term back-up pro- 
tection are covered by Records Disposition 
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Authorizations as is currently required for hard 
copy records. 

2.3 ARCHIVAL SERVICES 

The long-term preservation of selected, valu- 
able machine-readable records will require ex- 
panded archival services. Objectives ior im- 
proved archival services relale primarily to the 
State Archives' responsibility to select, acquire, 
preserve and make available State government 
records of enduring value. This strategic plan is 
designed to phase in these services gradually 
over the next five years. SARA will begin by ac- 
quiring and describing approximately 100 valu- 
able machine-readable data files already ap- 
praised by the Special Media Records Project 
staff. During the next five years, SARA will devel- 
op the capacity to accession, describe, and pre- 
pare for storage data files v/ith enduring value, 
locate adequate storage space, develop a tape 
monitoring and maintenance program, and pro- 
vide reference services for its machine-readable 
records holdings. 

OBJECTIVE 6: Develop the capacity of the 
State Archives to accession, describe, and make 
available for research use machine-readable 
records holdings. 

In order to ensure that selected, valuable 
records are preserved for future research, the 
State Archives must acquire the resources and 
develop the capacity to process and describe 
archival records in machine-readable form. It is 
absolutely essential to obtain access to data pro- 
cessing services during the first year of this plan 
in order to begin accessioning more than 100 
data files which have already been identified as 
archival. In order to develop the capability to ac- 
quire valuable EDP records on an ongoing basis, 
the Bureau of Archival Services, Collections 
Management Unit, will develop technical specifi- 
cations and procedures for the transfer of data 
files to archival storage from State agencies. 
Such guidelines will Indicate the formats that 
agencies must use to transfer data to the 
Archives and set minimum standards for docu- 
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mentation that must accompany archival data 
files. 

Based on the anticipated volume of acces- 
sions, the Collections Management Unit will also 
establish processing guidelines for archival data 
files. In the past, data archives hove followed ex- 
tensive verification procedures for all accessions. 
These procedures involve comparing print-outs 
of selected records against codebooks to deter- 
mine the accuracy of the data and documenta- 
tion, performing some simple statistical analyses 
to check for consistency of coding schemes, and 
taking corrective measures where indicated. 
Some data archives are reassessing the feasibili- 
ty of performing such a thorough analysis for all 
accessions. ^3 in establishing accessioning guide- 
lines, the Collections Management Unit will con- 
sider whether different levels of processing are 
appropriate depending on the value and unique- 
ness of the data, the costs of verification and pos- 
sible remedial actions, and anticipated research 
uses. During the planning period, the Collections 
Management Unit will also establish descriptive 
standards which will meet internal Archives' ad- 
ministrative needs to control archival data files 
and will conform to national standards for the ex- 
change of descriptive information about archival 
holdings. 

During the next five years, the Archives antici- 
pates a gradual increase in reference requests 
for machine-readable records. The number of 
data files acquired initially will be modest and 
researchers may be unaware that the State 
Archives preserves data in machine-readable 
form. Therefore, special efforts must be made to 
publicize the State Archives' data holdings. In 
order to encourage research use of the Archives' 
machine-readable record holdings, the Bureau 
of Archival Services, Reference Services Unit, in 
conjunction with the SARA External Programs 
Division, will submit notices and articles to the 



appropriate journals, and produce descriptive 
brochures, guides and other finding aids where 
appropriate. The Collections Management Unit 
will also enter descriptive information about 
SARA machine-readable records holdings into 
the appropriate national databases as the 
records are acquired and described. During the 
next five ^ears, the Reference Services Unit will 
also develop the capacity to meet researchers* 
requests as they occur. This will involve estab- 
lishing procedures for copying topes in conjunc- 
tion with a data processing service bureau and 
gaining familiarity wit'i descriptive materials for 
machine-readable records. Policies and proce- 
dures will also be developed to balance rights of 
access with the protection of confidential data, 
especially data which may affect personal privacy. 

OBJECTIVE 7: Develop the capacity of the 
State Archives to store machine-readable 
records and perform maintenance and preser- 
vation acli/ities for EDP records. 

Preservation activities for machine-readable 
records during the next five years v/ill be limited 
primarily to identifying and establishing appro- 
priate storage facilities and developing a tape 
preservation monitoring system. Locating ade- 
quate storage space for master and back-up 
copies of archival topes is a top priority because 
the Archives already has acquired a few data 
files and anticipates additional accessions in the 
coming year. The storage of magnetic tape in 
substandard storage areas increases the risk of 
data loss and will add to preservation costs in the 
future. Ideally, master tapes will be stored in a 
tape library associated with the data processing 
service bureau that the Archives selects. If the 
service bureau cannot provide secure and envi- 
ronmentally controlled storage, an alternative 
site will be required. This may require installa- 
tion of environmentally controlled vaults at the 
State Archives. In addition, an off-site storage lo- 



Archives of Canada, "Fi> e Year Plan for the Managing and Archiving of Machine Readable Data in the Pubkc Archives of 
Canada," Ottawa, 1986, pp. 56-57. 
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cation must be identified for back-up copies of 
tapes. The tape storage facilities at the Records 
Center will be evaluated for this purpose. Long- 
range plans for improved State Archives facilities 
must accommodate the storage requirements of 
special media records. 

Because the State Archives is just beginning 
to acquire machine-readable records, only basic 
conservation activities are anticipated during the 
next five years. Tope maintenance services, such 
as periodic rewinding of tope, checking for er- 
rors, and transfer of data to new tape will be pro- 
vided by a data processing service bureau. The 



Archives' Preservation Unit must develop a track- 
ing system to monitor tapes and to ensure that 
maintenance procedures are performed on a 
regularly scheduled basis. Such a system should 
be in place by the second year of the planning 
period to accommodate current and anticipated 
accessions. The Preservation Unit will also as- 
sume overall responsibility for investigating al- 
ternative storage media to magnetic tape. 
Although no cost-effective alternative currently 
exists, developments in optical disk technology 
and other storage media should be monitored as 
potential alternatives to magnetic tape storage. 
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PART THREE: 

SPECIFIC ACTIVITIES AND RESOURCE REQUIREMENTS 

OF THE FIVE-YEAR PLAN 



Part Three of this report outlines 27 specific ac- 
tivities thai must be undertaken during the next 
five years (FY 1988-89 to FY 1992-93) to integrate 
machine-readable records into State government 
records management programs and to develop 
the capacity at the State Archives to select, ac- 
quire and preserve records with enduring value 
in machine-readable form. The activities estab- 
lish a workplan to accomplish the objectives de- 
scribed in Part Two. As with the objectives, the 
activities are grouped into three categories 
which correspond to the three Bureaus in SARA*s 
State Government Records Division: Records 
Analysis and Disposition, Agency Services, and 
Archival Services. For each activity there is a 
time frame and an explanation of why each issue 
should be addressed and how each activity 
should be undertaken. Although each activity is 
assigned to a specific SARA unit, many activities 
will require assistance from State agencies and 
additional resources in agencies to improve their 
own management of EDP records. 

Summary of Objectives and Activities 

RECORDS ANALYSIS AND DISPOSITION: 
This Bureau guides the adequate documentation 
of New York State government, provides analysis 
and review of all Executive Branch records dis- 
position, coordinates approval of records disposi- 
tion schedules and requests with the Department 
of Law and the Office of the State Comptroller, 
and conducts archival appraisal of the records of 
all three branches of government. This plan sets 
two objectives for this Bureau: 1) to develop 
guidelines for the identification, description and 
retention scheduling of EDP records so that ma- 
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chine-readable records are included in the 
records retention and disposition review and ap- 
proval process, and 2) to ensure that agency pro- 
grams are adequately documented for legal, fis- 
cal, administrative, and historical purposes. 
Specific activities to accomplish these objectives 
include ilie development of guidelines and pro- 
cedures for scheduling EDP records, completion 
of a survey of centralized EDP systems, develop- 
ment of guidelines for identifying potential 
archival records in electronic for nats, a study of 
the implications of office automation systems for 
records keeping and records retention, and an 
evaluation of access policies for records in ma- 
chine-readable form. In addition, this Bureau will 
analyze and appraise EDP records as early as 
possible in their life cycle and conduct special 
appraisal studies of older and endangered ma- 
chine-readable records. The goal of this plan is to 
develop basic policies, guidelines and proce- 
dures by 1990 for EDP records produced by cen- 
tralized automated systems and to hove policies 
and guidelines in place for office automation sys- 
tems by 1992. 

AGENCY SERVICES: This Bureau operates the 
State Records Center for the storage of inactive 
records, provides other centralized services such 
as micrographics services, develops education 
and training programs in all aspects of records 
management, develops standards, and provides 
technical assistance in the use of technologies 
and in sound records management techniques. 
The objective of this plan is to ensure that SARA 
services in *hese areas meet the needs of agen- 
cies for electronic records management. Specific 
activities will include the development of educa- 
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tion and training mcterials on data management 
and retention scheduling of EDP records, the dis- 
semination of guidelines on tape maintenance 
and preservation, and the evaluation of stan- 
dards for tape storage and data interchange. 
This Bureau will also monitor new record keep- 
ing technologies, such as optical disk, and pro- 
vide assistance to agencies in use of these tech- 
nologies. To ensure that off-site storage for 
back-up tapes and inactive data files meet the 
special needs for maintenance and storage of 
EDP records, this Bureau will assess current con- 
ditions. The assessment may recommend 
improvements in storage vaults and additional 
tape maintenance and preservation services. 

ARCHIVAL SERVICES: This Bureau serves as 
New York State government's archival program. 
It accessions, arranges and describes, preserves 
and provides reference services for the perma- 
nently valuable records of New York State. 
Because New York State's permanently valuable 
records now include records in machine-read- 
able form, the objective of this plan is to ensure 
that the State Archives develops the capacity to 
acquire, preserve and make EDP records avail- 
able. During the first two years of the plan, the 
Bureau's Collections Management Unit should 
secure computer services for accessioning and 
processing EDP records, develop descriptive 
standards for these records, and obtain sufficient 
staff to handle accessioning and descriptive 
work. As the State Archives acquires permanent- 
ly valuable records in machine-readable form, 
the Bureau's Reference Services Unit should pub- 
licize their availability to researchers and devel- 
op the capacity to meet research requests. The 
Preservation Unit should ensure that data pro- 
cessing services are available to perform basic 
tape maintenance and preservation activities, 
and should investigate alternative storage media 
for electronic records. By the end of the planning 
period, the State Archives will need a carefully 



controlled storage area to house 150 to 200 reels 
of tape and to accommodate a collection of sev- 
eral thousand tapes in the future. Long-range fa- 
cilities planning must include provisions for a 
much larger volume of material, based on esti- 
mates that are revised as the Archives acquires 
more records in electronic formats. 

After the description of objectives and activi- 
ties for each program area, an estimate of re- 
source needs is provided. Estimates of resource 
needs at SARA include persoiiol services, sup- 
plies, equipment, and contract services. Resource 
estimates are also provided in tabular form at the 
end of the report. The resources needed for some 
activities cannot be estimated at this time be- 
cause more detailed siudies or needs assess- 
ments must first be completed. Moreover, this 
plan does not attempt to estimate the resources 
that agencies will need to improve management 
of their EDP records or to assist SARA staff with 
several of the activities outlined in the report. 
SARA will need assistance and advice from the 
NYS Forum for Information Resource 
Management, the Interagency Committee on 
EDP, the Governor's Office of Management and 
Productivity, and fronc other advisory bodies to 
ensure that policies and programs for managing 
electronic records meet the needs of agencies 
and of State government. Once overall pohcies 
and guidelines are in place, agencies must de- 
vote additijOnal resources to the analysis and re- 
tention scheduling of EDP records, to assess- 
ments of internal policies, and to staff training. 
Many agencies may need add.tional staff in both 
the EDP area and in records management to de- 
velop retention and dispooilion schedules and to 
better manage and preserve electronic records. 
SARA will need agency assistance to identify 
EDP records with archival value, to compile ade- 
quate documentation, and to transfer perma- 
nently valuable EDP records to the State 
Archives for preservation. 
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Description of Specific Activities 



Program Area: RECORDS ANALYSIS AND DISPOSITION 

Objective 1: Develop guidelines and assist all Executive Branch agencies to 
implement programs which identify* describe* and schedule the 
disposition of data from automated information systems 

Activity 1: Support and promote development of governmenZ-wide policies 

and guidelines for retention and disposition of records from au- 
tomated information systems. 



Time Frame: 
Explanation: 



1988-89 to 1989-90 
1990-91 to 1992-93 



Policy Development 
Implementation 



Activity 2: 
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During the past few years, New York Stale has recognized problems with the in- 
adequacy of its records management programs. Several coordinating bodies 
have been established and new initiatives are being undertaken to address 
these deficiencies. All Executive Branch agencies are participating in the 
Records Management Improvement Project sponsored by the Governor's Office 
of Management and Productivity. Recently, managers from many State agencies 
have joined together lo form the New York State Forum for Information Resource 
Management. The Forum promotes policies and practices for effective use and 
management of information resources in State government. In order to improve 
coordination between r^ ords management and archived activities, the State's 
records management and archival functions were consolidated with the transfer 
of statewide records manaaemont program responsibilities to the State 
Education Department. 

Efforts to improve records management and to establish effective information re- 
source management programs should be comprehensive. Guidelines, policies, 
and procedures for records management programs must include EDP records 
because an increasing portion of the State's public records exist in electronic for- 
mats. Information resource management programs must consider broad policy 
issues of access, use, dissemination, maintenance, and preservation of informa- 
tion in both electronic and hard copy formats. SARA should work with the 
Governor's Office of Management and Productivity, the NYS Forum for 
Information Resource Management, and the Interagency Committee on EDP to 
ensure that these efforts are comprehensive and coordinated. These groups 
should recommend government-v/ide policies where needed to improve man- 
agement and archival preservation of records from automated information sys- 
tems. 

Develop guidelines for records retention and disposition that in- 
tegrate EDP records into a revised review process under subdi' 
vision Ih Sec. 57.05 of the Arts and Cultural Affairs Law. 



Time Frame: 1 988-89 to 1 990-9 1 Development 

1 99 1 -92 to 1 992-93 Implementation 

Explanation: During the post two years, the Special Media Records Project has acquired some 

experience in the scheduling of data from automated systems. The project staff 
recommend that agencies approach the development of retention schedules for 
machine-readable records from a total information system perspective whenever 
possible. Agency EDP staff, program staff, and records officers should analyze 
their automated information systems to determine how long specific files, data 
bases, or data elements need to be retained, which formats are most appropriate 
for retention, how information will be maintained and made accessible during its 
retention period, and which records, if any, should be transferred to the State 
Archives for preservation. Disposition analysis should consider the retention of 
machine-readable and hard copy records together. Whenever possible, deci- 
sions regarding the retention, maintenance and disposition of data should be 
made during the design phase for new automated systems. 

While these general principles can aid the development of policies for schedul- 
ing records from automated systems, agencies need more specific guidelines for 
determining appropriate retention periods and identifying EDP records with po- 
tential archival data. Staff in the Bureau of Records Analysis and Disposition 
should draft guidelines for retention and preservation of EDP records. These 
guidelines might include a General Retention and Disposition Schedule for data 
processing records that ore common to all agencies and have short-term reten- 
tion requirements. Draft guidelines should be reviewed by agency records offi- 
cers, program managers, and EDP staff, and revised to address organizational, 
technical, and practical problems in the agencies. 

Once guidelines ore established, agencies should develop internal mechanisms 
to ensure that EDP records ore covered by adequate retention and disposition 
schedules. Scheduling EDP records will require additional resources in agencies 
for records management, systems analysis, and data management activities. 
These resources are not included in the resource estimates below. 



An essential element of improving the retention and disposition of EDP records is 
to ensure that records in electronic and other machine-readable formats are in- 
corporated into the records disposition review process. In the past, machine- 
readable records hove not been covered by disposition schedules or approved 
Records Disposition Authorizations (RDAs) as required by Subdivision 1 1, Sec, 
57,05 of the Arts and Cultural Affairs Low (formerly Sec. 186 of the State Finance 
Law). To remedy this situation, the parties which review Records Disposition 
Requests should insist that agencies develop retention schedules for their EDP 
records. 



Activity 3: Conduct a survey of centralized automated information systems 

in all Executive Branch agencies. 

Time Frame: 1988-89 

Explanation: The Special Media Records Project compiled an inventory of all automated infor- 

mation systems supported by the central EDP units in 19 Executive Branch agen- 
cies. This information provided Archives staff with an overview of the major data 
systems in each of the agencies and formed the basis for identifying priority 
areas for follow-up analysis and appraisal. 

During the current year, all agencies will complete an inventory of their records 
as part of the Governor's Records Management Improvement Project. The survey 
form, (OMP-1) asks each program unit to identify the data processing system or 
application, if any, used to create records that are stored in program units. 
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The inventory did not cx^llect infomiation about the cnitomated systems supported 
by central EDP Units. In order to fill in this gap, the SARA Bureau of Records 
Analysis and Disposition should sponsor a survey of centralized EDP systems (ex- 
cept in those agencies already surveyed by the Special Media Records Project). 
This inventory information, combined with information already collected for the 
Records Management Improvement Project, will allow agency records officers 
and SARA staff to identify centralized automated information systems and link 
hard copy input and output records to a system or application. The inventory in- 
formation would also support the development of a State government data in- 
ventory currently under consideration by the New York State Forum's 
Infomiation Clearinghouse Working Group. 

Activity 4: Develop and implement guidelines for the management, 

scheduling, and disposition of data in office automation environ- 
ments. 

Tune Frame: 1988-89 to 1989-90 Analysis 

1990-91 to 1992-93 Program Development and Implementation 

Explanation: In the past few years, records managers and archivists hove begim to study the 

impact of office automation systems (stand-alone word processors and docu- 
ment-based communications networks) on the management, retention, and 
preservation of records. Pilot studies indicate some significant differences be- 
tween office automation envirorunents and centralized data processing sys- 
tems. Because office automation systems are generally user-driven, responsi- 
bility for the retention and disposition of the records they create often rests with 
end users. 

The State Archives and Records Administration has not conducted research on 
the management or preservation of records from automated office systems in 
New York State agencies. Little is known about the extent to which office automa- 
tion systems are being used, the types of information created and stored in office 
automation environments, or the current policies, if any, regarding the manage- 
ment and disposition of records from these systems. 

During the first two years of the planning period, SARA will undertake a study of 
information management and records disposition practices in program units 
with major office automation installations. Such a study will determine whether 
documents that formerly were created only in hard copy are now produced in 
machine-readable form; assess changes in information management and dispo- 
sition practices; analyze the impact of data transmission capabilities on the revi- 
sion, distribution and retention of documents; and evaluate the indexing and re- 
trieval capabilities of the system. Selected data from the Records Management 
Improvement Project inventory will be used to assess the extent of office automa- 
tion in Executive Branch agencies. 

Based on this study, SARA staff will work with agency program managers and 
records managers to draft guidelines for management and retention of records 
created in office automation environments. Such guidelines will define various 
stages in the production of correspondence and policy documents; help agencies 
determine when drafts as well as final reports need to be retained in order to 
document the formation of poLcy; and recommend appropriate formats (ma- 
chine-readable, microform, hard copy) for long-term retention of information 
from office automation systems. The proposed guidelines will be tested in a pilot 
agency. 



^'^See especially the interim Reports of the Public Archives of Canada/Department of Communications Information 
Management Working Group, Interim Heporte, prepared by John MacDonald, chairman, Ottawa, Public Archives of 
Canada, June 1984 and September 1985. 



Based on the experience of this pilot project, general guidelines will be devel- 
oped for implementation in all Executive Branch agencies. Such guidelines will 
help agencies identify when disposition requests must be filed for the machine- 
readable OS well OS hard copy documents, and provide policies and procedures 
for implementing the disposition guidelines. 

Objective 2: Ensure that State government programs and operations are ade- 
quately documented for legaL fiscaL administrative, and histori- 
cal purposes. 

Activity 5: Assess the timing and manner in which records analysis and 

archival appraisals are conducted in order to encourage com- 
prehensive retention scheduling as soon as possible alter 
records are created. 

Time Frame: 1988-89 to 1989-90 

Explanation: Improved records management practices for New York State government agen- 

cies should be based on sound information managonent principles. Records and 
information managers recognize that, given the impermanence of the storage 
media on which data is stored, hardware and 5>oftware dependency issues, and 
the relationship between machine-readable and hard copy records, the retention 
periods for all records created by automated systems should be established as 
early as possible in the record's life cycle. 

During FY 1988-89 and 1989-90, staff of the SARA Bureau of Records Analysis and 
Disposition will reevaluate the timing of records analysis and appraisal studies. 
This analysis will cover both EDP and hard copy records and assess the practical 
problems of evaluating records as soon as possible after they aie created. SARA 
staff will develop procedures to ensure a more timely analysis of nev/ records 
systems as they are designed, including mechanisms for learning about new in- 
formation systems before they are implemented. 

Activity 6: Develop guidelines {or the identification and appraisal of EDP 

records with potential archival value and identify priority ap- 
praisal projects. 

Time Frame: 1988-89 to 1989-90 

Explanation: The Special Media Project staff gained considerable experience in evaluating 

the archival value of EDP records. The Project's Preliminary Report estimated 
that as many as 35 percent of the major, centralized EDP systems may produce 
some data with archival value. The Report also suggested that survey files, regis- 
tration systems, case management systems, license-permit systems, and some 
general operational systems were most likely to produce data of archival value 
because they contain detailed social and demographic information about indi- 
viduals affected by State programs; information about important economic, so- 
cial and scientific phenomenon: and information that documents the p)olicies and 
programs of State government. 

Five major EDP systems were analyzed in detail by Project staff to determine pre- 
cisely which data files, if any, contained information of sufficient historical or 
other research value to warrant long-term preservation in machine-readable 
form. 



^^The systems studied were the Stale Education Department's Basic Educational Data System, the New York Stale 
Economic Database in the Department of Commerce, the Computerized Criminal History System (CCH/OBTS) in the 
Division of Criminal Justice Services, the Vital Records System in the Department of Health, and the Division of 
Equalization and Assessment's Private Utility and Industrial Property Valuation System. Copies of these studies are 
available on request from the State Archives and Records Administration. 



Thorough analysis of the systems enabled the Project staff to better estimate the 
amount of appraisal work needed to identify and preserve value archival 
records in madiine-readable form. After appraising five systems, the Project staff 
concluded only three produced data which should be preserved by the State 
Archives. This suggests that between 15 and 20 percent of centralized EDP sys- 
tems might produce archival records, rather than 35 percent as originally esti- 
mated. 

Given the extensive use of computer technology by State government agencies, 
it is important to stress that this is still a very large number of systems. Possibly as 
many as 150 to 200 centralized EDP systems produce some data with archival 
value. Therefore, guidelines ore needed to identify those systems that ore most 
likely to produce valuable data for future research. 

Based on the Special Media Project studies, it appears that data files created and 
used by agencies for their own internal research purposes ore the most likely to 
have future research applications. This includes survey files and special re- 
search files v/hich are often extracted from larger database for research and poli- 
cy evaluation by agencies. 

Even with additional appraisal staff, only a small percentage of the EDP systems 
which may produce archival records can be appraised during the next five 
years. Therefore, during the first year of this plan, a list of top priority systems for 
appraisal will be identified. In establishing priorities, SARA staff will take into ac- 
count the uniqueness of the data, its age, the extent to which it may be endan- 
gered in its current environment, and the likelihood that a detailed study will 
yield either valuable archival data files, major records management improve- 
ments, or both. This list will be used to allocate staff for special appraisal projects 
throughout the planning period and may provide the basis for a special project 
to preserve selected, valuable EDP records. Agencies need additional guidance 
to help Archives staff identify other criteria which might be used to identify ma- 
chine-readable records with potential archival value, SARA staff should develop 
a set of general guidelines for identifying EDP records with potential long-term 
value. This information will be provided to agencies to assist them with schedul- 
ing their EDP records. General guidelines for identifying EDP records with long- 
term value, however, will not eliminate the need for thorough analyses of dispo- 
sition schedules for major EDP systems by staff of the Bureau of Records Analysis 
and Disposition. 

As agencies begin to submit retention and disposition schedules for EDP records, 
it is likely that a significant number of valuable, endangered machine-readable 
records will need immediate appraisal and timely transfer to the State Archives 
for preservation. Based on the results of initial invoitory and records scheduling 
activities in agencies, a special initiative may be needed at SARA, beginning in 
1990-91, to identify and preserve selected, vcduable EDP records that ore in dan- 
ger of loss or deterioration in State agencies. 

Activity 7: Conduct special appraisal studies of several major information 

systems each year. 

Time Frame: Ongoing 

Explanation: To accomplish the State Arcbves' mission to identify and preserve records of last- 

ing value for historical and other research, the Bureau of Records Analysis and 
Disposition must develop the capacity to appraise machine*readable records 
from several major automated systems each year. Specific systems to be ap- 
praised will be selected from the list of priority appraisal projects (see Activity 6) 
and based on agency records management improvement plans and schedules 
submitted by agencies. 
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Activity 8: 



Include machine-readable records in special initiatives to im- 
prove records management and archival programs in the 
Legislative and Judicial branches of government* 



Tune Frame: Ongoing 

Explanation: "The Legislative and Judicial branches of government face records management 

problems for both traditional files and machine-readable records that are similar 
to those encountered in Executive Branch agencies. The Senate, the Assembly, 
and the Unified Court System rely heavily on automated information systems for 
records creation, retrieval, and storage. However, no comprehensive guidelines 
exist to promote records management or ensure archival preservation of valu- 
able records in these branches. 

Two initiatives are under way to address records management and archival 
problems in the Legislature and the Courts. Since 1984, the State Archives has 
worked with the Office of Court Administration on a Judicial Records Project 
which has resulted in records retention schedules for all of the State's major 
types of courts. During the next five years, as these schedules are tested and im- 
plemented in court sites, OCA records managers will likely encounter and need 
to address issues and problems with EDP records which parallel those of the 
Executive Branch. 

A preliminary assessment of archival and related records management needs in 
the Legislature, begun by SARA, in 1987, indicates that the use of computers is 
widespread. In the Assernbly, for example, all assemblymen hove personal com- 
puters for producing correspondence. Offices, such as the Task Force on 
Demographic Research and the Legislative Bill Drafting Commission, appear to 
produce important records in machine-readable form. In order to ensure that the 
full range of records management and arphival issues is addressed in the 
Legislature, machine-readable records must be included in SARA's plans to 
identify and preserve valuable legislative records. 



RECORDS ANALYSIS AND DISPOSITION: SUMMARY OF RESOURCE NEEDS 

•During the first two years, the Bureau Chief will devote 15 to 20 percent time to 
coordinating and supervising the development of data retention guidelines* 
and to revising the records disposition review cmd approval process to accom- 
modate H)P records. In addition, the Bureau Chief will consult with the Head 
of the Collections Management Unit in the Bureau of Archival Services on ob- 
taining adequate data processing services and on developing accessioning 
and descriptive procedures for archival mach5 -^-readable records. 

One additional Associate Archivist is needed In this Bureau to develop guide- 
lines for identification and appraisal of EDP records with potential archival 
value and to conduct appraisal studies of valuable and endangered EDP 
records. This Archivist will also assist the Collections Management Unit wltJ 
transfer to the Archives of more than 100 data flies that were Identified as 
archival by the Special Media Project. During FY 1988-89 SARA should fill a va- 
cant temporary services position for an Associate Archivist. This position 
should be transferred to permanent State fimdlng beginning in FY 1989-90. 
Approximately 25 to 30 percent of the time of an Associate Records Analyst Is 
also needed to develop guidelines for retention and disposition of EDP records 
and to conduct a survey of centralized automated Information systems. The 
Associate Archivist for Legislative Records will spend approximately 10 per- 
cent time evaluating the records keeping and archival implications of EDP 
records in the Legislature. Beginning In 1990, this Bureau will need one addi- 
tional Associate Analyst to develop guidelines for records produced by office 
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automation systems* Furthermore^ all professional staii in the Bureau will re- 
view, analyze* and appraise records schedules and disposition requests from 
agencies thai produce EDP records* Additional resources may be needed be- 
ginning in 1990 to appraise valuable data thai is in danger of loss or destruc- 
tion* 




Program Area: AGENCY SERVICES 



Objective 3: 

Activity 9: 

Time Frame: 
Explanation: 



Activity 10: 



Increase awareness among records managers* program man- 
agers* EDP specialists and users of the primary and secondary 
values of EDP records* and of policies regarding access* reten- 
tion* disposition* and long-term preservation of data. 

Evaluate government-wide policies regarding access to records 
to ensure that data in automated information systems is avail- 
able and that confidential data is protected. 



1988-89 to 1989-90 
1990-91 to 1992-93 



Development 
Dissemination 



Access to public records is currently regulated by tlie Freedom of Information 
Law (Article 6, Public Officers Law), the Personal Privacy Protection Law (Article 
6A| Public Officers Law) by specific statutes and administrative rules, and, in 
some coseSi by federal laws and regulations. These regulations apply to all pxib- 
lic records regardless of physical format or recording medium. More precise 
guidelines may be needed to provide access to records in machine-readable 
form. Special procedures may also be needed to protect confidential data be- 
cause such information is easily duplicated and unauthorized access to comput- 
erized data banks may be difficult to trace. The complex flow of information 
among State agencies and between the State and federal level also raises juris- 
dictional issues. 

A cooperative assessment of existing access, confidentiality and security regula- 
tions and procedures for machine-readable records is needed to determine 1) 
v/hether existing policies and procedures are adequate to protect confidential in- 
formation and to provide access to information that is not restricted; 2) whether 
access and privacy provisions are consistent among agencies and between the 
State and federal level; 3) whether current policies and procedures provide suffi- 
cient guidance to agency staff who must administer requests from the public to 
access data in automated information systems; 4) what provisions, if any should 
be made regarding the transfer of machine-readable records that contain confi- 
dential data to the State Archives; and 5) what provision should be made to pro- 
vide researchers with restricted access to confidential data. 

Broad questions such as the definition of a "record" in a database environment 
and the status of software developed in-house by State agencies must be ad- 
dressed. An ossessrrient of security provisions should include the impact of in- 
creasing decentralization of data processing on the physical security and privacy 
protection of data. 

Once current policies hove been assessed and gaps and inconsistencies identi- 
fied, an interagency v/orldng group, including staff from SARA and from the 
Committee on Open Government, should develop remedies. These may include 
changes in the existing law, further development and monitoring of existing pro- 
cedures, more precise guidelines for agency staff, and other activities where ap- 
propriate. Once consistent p)olicies are in place, this information should be dis- 
seminated to program officers, records managers, and EDP units through SARA 
and agency training programs. 

Develop and disseminate training materials on the manage- 
ment, preservation, and uses of EDP records to records man- 
agers, program managers, and EDP units. 



Time Frame: 



Ongoing 
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Explanation: 



Objective 4: 

Activity 11: 

Time Frame: 
Explanation: 



Activity 12: 

Time Frame: 
Explanation: 



To expand and improve tlie Slale*s records management program, SARA will 
provide training to records officers and program managers. In order to integrate 
management of EDP records into this program, records officers need training in 
the management and scheduling of machine-readable records, Sp>ecial initia- 
tives are also needed to provide training in the management of EDP records to 
data processing staff. 

In fiscal year 1988-89, SARA v/ill assess training needs and develop training pro- 
posals for all types of records management activities. To ensure that SARA edu- 
cational programs adequately cover training needs for both EDP and paper 
records, the SARA Training and Education Coordinator should develop work- 
shops and other training materials addressing the issues and techniques of EDP 
records management. Training in data management and EDP records schedul- 
ing should be disseminated to agencies as an ongoing element of the SARA 
training and education program. 

SARA should also use consultants and contract services for state-of-the-art pre- 
sentalioris on new issues in records and information management. Likely areas 
for training in the later years of the plan include electronic mail, optical disk tech- 
nology, and managing information in office networks. 

Enhance the ability of State agencies and SARA to manage and 
preserve records created with modern information technology. 

Integrate guidelines for scheduling EDP records into inlormation 
systems design methodologies. 



1989-90 to 1991-92 
1992-93 



Development 
Implementation 



Archivists, records managers, and other information specialists agree that provi- 
sions for the management and long-term preservation of EDP records should be 
an integral part of systems design methodologies. Just as current automated sys- 
tems planning includes specifications for the appropriate hardware and software 
to meet immediate information needs, systems pic^^-^ing should also include a re- 
tention plan for the products of automated information systems. Using this ap- 
proach, agencies can develop comprehensive records retention plans for all of 
the records associated with automated systems. Automated systems can be de- 
signed to handle the disposition of information more or less automatically by 
deleting data when no longer needed for legal, fiscal, administrative or research 
purposes and by creating those records with archival value in a format appropri- 
ate for transfer to the State Archives (See Activity 18). Retention plans, once ap- 
proved as required by law, would also bring records created end stored in ma- 
chine-readable form into compliance with legal requirements for approval prior 
to the destruction of public records. 

Beginning in FY 1989-90, the SARA Records Management Specialist for 
Technology and Standards vrill work with an advisory group of EDP spcialists 
and agency records managers to develop guidelines to integi^ate the retention 
scheduling of EDP records into information systems design procedures. After re- 
view and testing by agencies, these guidelines should be applied to new auto- 
mated information systems as they are designed. 

Develop policy guidelines and procedures for preservation of 
dafa by agencies over extended periods of fime. 

1989-90 to 1991-92 

Some machine-readable records do not have permanent value, but still need to 
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be retained for extended periods of time to meet agency administrative, fiscal, 
and/or legal requirements. Machine-readable media, however, ore subject to de- 
terioration after a relatively short period of time. When agencies need to retain 
machine-readable records for more than five years, special storage and mainte- 
nance prrxedures should be followed in order to increase the longevity of data 
files and ensure that data is available throughout the specified retention period. 

The SARA Specialist for Technology and Standards, in conjunction with selected 
agency EDP \inils, should develop a set of storage and maintenance procedures 
which will help agencies preserve machine-readable records to meet their inter- 
nal information requirements. Such guidelines will specify proper temperature 
and humidity requirements for long-term storage, identify where adequate stor- 
age facilities ore available, describe maintenance routines, encourage proper 
back-up procedures, and indicate when data stored on topes, diskettes and oflier 
magnetic media should be copied to new media to ensure long-term access to 
the information. The guidelines will also address the need to ensure that software 
necessary to retrieve data in inactive storage is available throughout the speci- 
fied retention period for the records or to reformat records so that they can be re- 
trieved without access to special software. 

Activity 13: Monitor technological developments in inlormation systems and 

storage technology 

Time Frame: Ongoing 

Explanation: Innovations in information and storage technology promise to provide new and 

cost-effective ways to store and communicate information created and used by 
State government agencies. However, such changes will have a profound impxact 
on the management and archival preservation of records. In dealing with major 
changes in information technology in the past, records managers and archivists 
hove taken a reactive stance, developing solutions only after significant man- 
agement and preseivation problems were apparent and important documenta- 
tion was lost. 

In order to respond to ongoing changes in information technology and to assess 
their likely impact on records management and archival programs, the SARA 
Technology and Standards Specialist will monitor new information technology. 
'Technology assessments will address two aspects of new information technolo- 
gies: 1) their potential use for improved records creation, storage and retrieval, 
and 2) their possible implications for the archival preservation of selected, valu- 
able government records. Office automation systems, electronic modi, optical disk 
technology, and data transmission systems are priority areas for assessment in 
the initial years of the plcoming period. 

Activity 14: Promote government-wide procedures and mechanisms lor data 

interchange between hardware and software systems by evalu- 
ating standards for the transfer of data from one computer sys- 
tem to another. 



Time Frame: Ongoing 

Explanation: The incompatibility of computer hardware and software systems creates a vari- 

ety of problems for data administrators, users, records managers, and archivists. 
Systems incompatibility exists on three levels: hardware, operating systems, and 
applications software. Hardware incompatibility is most prevalent in microcom- 
puter-based systems and office automation networks. The use of diskettes of dif- 
ferent sizes and disk drives which write data in various formats makes the ex- 
change of data from one hardware system to another impossible in many coses. 
Incompatible operating systeiris ore a further obstacle to data interchange 
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among systems. In office cmtomation environments this also contributes to a pro- 
liferation of applications software and to the creation of data in formats that can- 
not be easily passed from one system to another. In mainframe-based systems, 
software incompatibility is especially common where hierarchical or relational 
database management systems are used. 

Hardware and software incompatibility creates data interchange problems for 
current users of records. When two users wish to exchange information electroni- 
cally typically they must hove access to the some or compatible hardware and 
software systems, or the data must be reformatted so that it con be accessed and 
read by the receiving system. The implications of systems incompatibility are 
also critical for records managers and archivists. li data are to remain accessible 
in machine-readable form over extended periods of time, archivists and records 
managers must ensure that the hardware or software necessary to access and 
retrieve the data will be available with the data. Most data archives have dealt 
with this problem by preserving data in a format that is as hardware-end soft- 
ware-independent as possible (generally as numeric data in a sequential file 
with fixed-length fields and records). 

Archivists and records managers are reassessing the problems of hardware- 
and software-dependency. Downloading data from a database management 
system into flat files usually is a costly undertaking because it requires extensive 
analysis and programming, especially where utility packages are not readily 
available to download data. Moreover, reformatting data from a database envi- 
ronment designed to provide direct access to various data elements and records 
into a format where data can only be accessed sequentially con significantly re- 
duce the usefulness of the data. Finally, the flat file format is inappropriate for 
many systems that store non-numeric data, such as cartographic databases, 
computepdssisted design systems, and text retrieval systems. 

Some archivists and records managers are exploring alternative formats for the 
preservation of machine-readable data. One approach is to preserve the soft- 
ware necessary to access a database or a te'1 file along with the data. This ap- 
proach has generally been considered impractical because of the cost of main- 
taining software which may be used only occasionally. However, under certain 
circumstances, especially in the area of common micro-computer software pack- 
ages, the costs of software maintenance may be justified if a large volume of 
records with long-term value are created in these formats. 

A more promising alternative is t^ie use of data interchange standards which are 
currently being developed by various national and international standards bod- 
ies. The ISO reference model, for example, a developing standard for data inter- 
change between mainframe systems, shows promise of promoting standard in- 
terfaces which would allow exchange of data without reformatting between 
incompatible systems. Interchange fonnats hove also been proposed and adopt- 
ed by numerous federal government agencies for the interchange of textual doc- 
uments between office automation systems. The EDP community, recoixls man- 
agers, and archivists should evaluate the utility of these standards and 
investigate various means to promote their adoption and use by State govern- 
ment agencies. 

Objective 5: Evaluate the role of the State Records Center in providing data 
storage, security and preservation. 

Activity 15: Assess the qualitY ond capacity of storage areas for EDP 

records, including their utility lor data security and vital records 
protection. 

Time Frame: 1988-89 to 1989-90 Assessment 

1 990-9 1 to 1992-93 Make improvements as indicated 
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Explanation: Currently the State Records Center provides off-site storage for back-up files and 

inactive magnetic tapes. In FY 1987-88 tlie Records Center handled approximate- 
ly 150,000 reels of tape. This activity is growing at a rate of about 20 percent per 
year. In addition, approximately 22,500 inactive tapes are stored at the Records 
Center In order to ensure that these records are stored under proper conditions, 
an assessment of the environmental quality of the storage areas should be un- 
dertaken. Magnetic tapes should be stored at as constant a temperature and rel- 
ative humidity as possible. Current standards recommend a storage temperature 
of 65 degrees F with fluctuations in storage areas of no more than (+ or -) 5 de- 
grees, and a relative humidity of 4b percent with a fluctuation of no more than (+ 
or -) 5 F>ercent. Studies of taF>e deterioration indicate that storage at a constant 
temperature and humidity are more important than base temperature and hu- 
midity levels because fluctuations in temperature and humidity ore most damag- 
ing to tapes. Temperature and humidity in the tape vaults should be monitored 
over a one-year period to ensure that the storage conditions meet basic quality 
standards. Security systems should also be evaluated to ensure that vital data is 
secure and that the possibility of unauthorized access to restricted data is mini- 
mized. An evaluation of this service should also review the capacity of the 
Records Center to handle a rapidly increasing amount of activity. Based on an 
assessment of these problems during the first year of the planning period, appro- 
priate measures should be undertaken to remedy any problems uncovered by 
the evaluation. 



Activity 16: Determine whether the Records Center should provide tape 

maintenance and/or conservation services for EDP records with 
retention periods in excess of five years. 

Time Frame: 1988-89 to 1989-90 Assessment 

1990-91 to 1992-93 ImplementaUon 

Explanation: Regular maintenance procedures are needed to ensure the readability of data 

when magnetic tapes are stored for periods of more than five years. Tapes in in- 
active storage should be inspected periodically, rewound under constant tension, 
and transferred to new magnetic media to ensure the long-term readability of the 
data. Currently these activities, if conducted at all, are the responsibility of the 
agencies which create or produce the data. During the first two years of the plan- 
ning period, a feasibility study of the need for centralized services for tape main- 
tenance and conservation will be conducted to determine whether it would be 
cost-effective for SARA to provide such services on a centralized basis. Pending 
the outcome of the initial assessment, further development of tap>e cleaning, in- 
spection, and copying services may be developed. 



ftGENCY SERVICES: SUMMARY OF RESOURCE NEEDS 

During the first two years, on evaluation of tape storage conditions and an as- 
sessment of needs for centralized tape maintenance and preservation will re- 
quire approximately 10 percent of the time of the Bureau Chief, The 
Coordinator for Training and Education should spend approximately 15 per- 
cent time throughout the planning period on needs assessment, and on the de- 
velopment and procurement of training materials for EDP records manage- 
ment. In addition, approximately $5,000 of contract service funds should be 
devoted each year to bringing exports in technology issues to Albany to pro- 
sent seminars and workshops for State agency staff. 

When SARA develops Technology and Standards services beginning in 1988- 
89, the Coordinator should devote at least 10 percent time to monitoring techno- 
logical developments in the area of data processing and investigating data in- 
terchange standards. In tho last four years of the plan, this need will increase 
to approximately 40 percent time for the Coordinator to assist EDP units with 
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implomonting data retention guidelines and to develop tope maintenance and 
preservation guidelines. The estimates for resources needed by ihe Agency 
Services Bureau do not include the costs of possible capital improvements to 
Records Center storage vaults and environmental monitoring systems* or the 
costs of providing tape maintenance and preservation services. 



Program Area: ARCHIVAL SERVICES 



Activity 17: 

Time Frctme: 
Explcmation: 



Activity 18: 

Time Frame: 
Explanab'on: 



Activity 19: 

Time Frcame: 
Explanation: 



Develop the capacity to accession^ describe/ and make avail- 
able for research use holdings of machine-readable records. 

Obtain data processing services and resources needed to acces- 
sion machine-readable records identified as having archival 
value. 



The preparation of data Ii'<*s for long-term storage will require access to a com- 
puter center that con provide a variety of data processing services. The computer 
center must hove the capacity to read and process data produced in a variety of 
outside EDP units, to copy and reformat files, to produce printouts of selected por- 
tions of files for verification purposes, and to provide access to common statistical 
packages. Consultative services to assist the Archives staff with such problems as 
unreadable tapes, coding errors, and software dependency ore also desirable. 

Currently, the State Education Department's EDP Division is providing the 
Archives with computer access and technical support for the transfer of archival 
data files from two agencies to the State Archives. During FY 1988-89, the State 
Archives and the EDP Division will use these pilot projects to evaluate this ar- 
rangement for data processing services. These projects will also enable SARA 
and the EDP Division to develop more accurate estimates of the resources need- 
ed to acquire machine-readable records on a continuing basis. If the 
arrangement with the EDP Division is unsatisfactory, SARA will investigate alter- 
native sources of EDP services in FY 1989-90 and select a data processing center 
for processing of archival data files no later than April 1, 1990. 

Develop technical specifications and procedures for the transfer 
of machine-readable records to the State Archives. 

1988-89 tolP89-90 

During the first two years covered by this plan, the State Archives will establish 
standards for the transfer of archival data files from agencies to the Archives in 
consultation with agency records management and EDP staff. These standards 
will specify the storage media, densities, labeling conventions, preferred file 
structures, and minimum acceptable file documentation. Additional standards 
will b© proposed later in the planning period for data created on microcomput- 
ers, in automated office systems, and for noncdphanumeric data. 

Develop processing guidelines, including appropriate levels of 
processing for archival data files and assign responsibility for 
various processing activities. 

1988-89 to 1989-90 

The transfer of archival data files from agencies to the State Archives requires 
careful preparation of the files for long-term storage and preservation. At a mini- 
mum, a printout of selected records from each file must be compared with the 
documentation to ensure that codebooks and file layouts match the data provid- 
ed and that coding schemes are documented adequately to allow future users to 
understand and interpret the data. In some coses, more elaborate file verifica- 
tion is needed, especially when the initial evaluation of the file uncovers prob- 
lems such as missing data, coding errors, or inadequate documentation. 



31 



ERIC 




AciivitY20: 

Time Frame: 
Explanation: 



ActiviiY 21: 

Time Frame: 
Explanation: 



ActivnjT22: 
Time Frame: 



Verification of data and the correction of errors are labor-intensive aclivitioo. 
Therefore, precise guideline regarding the appropriate level of processing for 
data files ore needed. Such guidelines should take into account the value and 
uniqueness of the data and the likely costs of remedying errors in the file or docu- 
mentation. Guidelines ore also needed to determine which levels of staff 'vill be 
assigned responsibility for verification and other accessioning procedures. 

Accession and process data from several automated inlormation 
systems each year. 



The Preliminary Report of the Special Media Records Project and follow-up ap- 
praisal ctudies hove identified more than lOO data filefi with long-term research 
value. Li order to establish a program to preserve this data and make it available 
to researchers, the State Archives will accession machine-readable files from 
several automated information systems eaeh year. As part of the State- AiJiives 
responsibility to pr^^^erve records with enduring value, data files will be trans- 
ferred to the Archives as part of the normal accessioning work flow. SARA antici- 
pates that the volume of accessions will continue to increase as agencies develop 
retention and disposition schedules which include machine-readable records 
and as SARA staff identify files that should be transferred to the Archives for 
preservation. Therefore, resource requirement projections are tentative and con- 
servative. Additional staff, supplies and services will be required if apprairal 
archivists identify large volumes of data files in immediate need of transfer to the 
Archives- 

Evaluate descriptive standards for machine-readable records in- 
cluding an assessment of the applicability of the Machine- 
Readable Cataloging, Archives and Manuscripts Control (MARC 
AMC) and Machine-Readable Data File (MRDF) formats for the 
description of EDP records. 



The MARC (Machine-Readable Cataloging) format is a standard format used by 
libraries and archives for bibliographic information. Numerous archives, includ- 
ing the New York State Archives, have adopted the MARC format for describing 
their holdings. Use of the MARC format enables repositories to enter descriptive 
data about their holdings into national bibliographic databases and to exchange 
bibliographic and other descriptive information with libraries, archives, and 
other repositories. Currently, seven slightly different versions of the MARC format 
are available for use with different types of materic's, including the MARC 
Archives and Manuscripts Control format (AMC) and the MARC format for ma- 
chine-readable data files (MRDF). Machine-readable records con be described 
in either or both of the formats and the archival community has not yet deter- 
mined which format is preferable. During the initial year of the planning period, 
the staff of the State Archives will participate in an assessment of the applicabili- 
ty of the AMC and MRDF versions of the MARC format. This assessment will be 
carried out in conjunction with other data archivists through the Society of 
American Archivist's Task Force on Automated Records and Techniques. 
Following the assessment, standards and procedures will be developed for the 
description of EDP records in the State Archives. 

Enter series descriptions for machine-readable records into one 
or more national databases as the records are processed. 
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Explanation: Archival repositories are beginning to distribute and exchange information 

about their holdings with libraries and other repositories through national 
databases, such as the Research Libraries Information Network (RUN) databases 
and the On-Line Computer Library Center's (CXULC) catalog. The State Archives 
currently is entering descriptive data about its holdings into the RUN database. 
Use of such national online catalogs can increase researchers' awareness of 
archival holdings because researchers who might not normally contact an 
archives directly about holdings may discover valuable records in the course of 
a search at a library or another repository. This is especially true for madiine- 
readable data files, because few potential researchers are aware that archival 
repositories collect and make available data in machine-readable form. 
Therefore, it is essential to enter descriptive information about machine-readable 
records in the State Archives into national databases as the records are pro- 
cessed. 

Activity 23: Produce descriptive brochures, articles \n appropriate journals, 

special guides, and other materials to inform potential users of 
the State Archives^ machine-readable I'ecords holdings. 

Time Frame: Ongoing 

Explanation: Special initiatives may be needed to encourage use of the State Archives ma- 

chine-readable records holdings by researchers. Without publicity to the appro- 
priate research communities, including researchers and' policy analysts in State 
government, most potential researchers will be unaware that the State Archives 
acquires machine-readable records. Moreover, the types of data files acquired 
by the State Archives will be of interest to a wider research community than that 
traditionally served by the State Archives. Machine-readable records are more 
likely to hove value for research in a wider variety of subject areas such as r ^o- 
lic administration, policy analysis, social services, criminal justice, medicine nd 
science. Therefore, the Reference Services Unit, with assistance from the iARA 
External Program Office, will prepare special notices, subject brochv^es, and 
similar publicity materials to ensure that the research community is aware of the 
data files collected by the State Archives. 

Activity 24: Meet researchers^ requests as they occur. 

Time Frame: Ongoing 

Explanation: Providing reference services for machine-readable records differs considerably 

from traditional paper files. Machine-readable data can be copied and provided 
to researchers at a minimal cost. Therefore, researchers can obtain data without 
the need to visit an archives. Often research requests are handled by providing 
an interested researcher with the codebook, user's guide and other descriptive 
materials for a file. If a data file will meet the researchers needs, arrangements 
are usually made to provide the researcher with a copy of the file for use at a 
computer center. In order to allow researchers to use the State Archives holdings, 
the Reference Services Unit must develop a set of access policies and proce- 
dures. Such policies should- address the degree of service which the State 
Archives will provide, the limitations on access to data which are restricted for 
reasons of privacy or confidentiality, and the need, if any, to assess user's fees for 
copies of data files. 

Objective 7: Develop the capacity to store machine-readable records and 
perform maintenance and preservation activities for EDP 
records. 

Activity 25: Secure adequate storage space for one master and one back-up 

copy of each machine-readable data file. 
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Time Frame: 
Explanation: 



Activity 26: 

Time Frame: 
Explanation: 



Activity 27: 

Time Frame: 
Explanation: 



Current estimates indicate that the State Archives will acquire approximately 180 
reels of tape (master and back-up copies) over the next five years. The storage of 
magnetic tape requires carefully monitored and strictly controlled storage areas 
with OS little temperature and humidity fluctuation as possible. Data archives 
generally maintain one master and one duplicate back-up copy as security in 
the event of tape deterioration. Master and back-up copies should be stored in 
two different locations. These special requirements should be token into account 
in identifying and selecting storage areas for magnetic tapes during Uie next five 
years. 

The State Archives currently is developing a long-range plan for storage of 
OTchival records in all formats. The long-range storage plan must include provi- 
sions for the storage of master and back-up copies of magnetic tapes. This plan 
should assess the feasibility of a shared facility with the State Records Center 
(See Activities 15 and 16). 

Obtain access to data prcr.essing services necessary to perform 
maintenance and preservation activities. 

1988-89 to 1989-90 

The long-term preservation of machine-readable records requires carefully mon- 
itoring of tape deterioration, regular maintenance routines, and occasional trans- 
fer of data to new magnetic tape. These services should be performed by a ser- 
vice center rather than done in-house by the State Archives staff. One criterion 
for selecting a data processing service center for the State Archives will be its ca- 
pacity to provide tape maintenance, conservation, and copying services. The 
Preservation Unit will assume responsibility for monitoring these services and 
design a system to monitor tape maintenance activities. (See Activity 17). 

Investigate alternative storage media. 



Currently, magnetic tape is the standard storage medium used for the long-term 
retention of machine-readable data because it is less expensive and more stable 
than most available alternatives. However, magnetic tape is not a permanent 
medium and its use for storage of machine-readable records requires regular 
maintenance and periodic recopying to new tape. Archivists remain optimistic 
that new developments in storage technology may yield a more stable and cost- 
effective medium for the permanent retention of data ip machine-readable form. 

Optical disk technology may eventually provide an alternative storage medium 
for electronic records, although the archival properties of optical disk technology 
hove not been fully tested. The storage of data on microfilm that can be scanned 
and reconverted to digital form also holds some promise. In order to take advan- 
tage of these technological developments, the Preservation Unit should monitor 
new developments in storage technology. 

In fiscal year 1989-90, Preservation Unit staff will begin an assessment of optical 
disk technology for archival storage with an extensive pilot project to test the opti- 
cal disk system currently under development at the National Archives in 
Washington, D.C. Additional resources may be requested for further develop- 
ment of optical disk systems at SARA, pending the results of the National 
Archives' pilot project. 
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ARCfflVAL SERVICES: SUMMARY OF RESOURCE NEEDS 



THe Principal Archivist who heads this BureoiL should spend 5 to 10 percent 
time r^ordinating and supervising the development of accessioning proce- 
dures, descriptive standards, a tope preservation and maintenance program, 
and reference services for EDP records. In the Collections Management Unit, 
the Unit Head will need 15 percent time in 1988-89 and 10 percent in 1989-90 to 
obtain data processing services for accessioning EDP records, establishing do* 
scription guidelines and procedures, and supervising the acquisition of ap- 
proximately 100 data files already identified as orchivoL 

One additional Senior Archivist is needed in the Collections Management Unit 
to accession and describe machine-readable records* During FY 1988-89, SARA 
should fill a temporary service-funded vacancy for this activity* Beginning in 
FY 1989-90 this position should be transferred to permanent State funding as 
pari of SABA*s ongoing archival program* In addition/ SARA will need approxi- 
mately $22,000 in contract service fimds each year for technical assistance, and' 
for data processing services and supplies for accessioning machine-readable 
records* 

The Head of the Preservation Unit should devote approximately 5 percent time 
in the first two years of the plan to the development of a tape preservation 
monitoring system and to an assessment of data processing services to ensure 
that services arc available for tape maintenance and preservation procedures* 
In addition, SARA must procure adequate facilities for the storage of topes* 
SARA should again request $150,000 in its 1989-90 budget for installation of 
vaults for storage of computer topes and other special medio records* Long- 
range facilities planning must include provisions for expanded storoge oreos 
for machine-readable and other special medio records* Finolly, the 
Preservotion Unit should request $60,000 in funding for o project to ossess the 
opplicobility of optical disk technology, beginning in FY 1989-90* This project 
will be corried out in conjunction with the U*S* Notional Archives and Records 
Administrotion which hos invested over $1 million in on optical disk pilot pro- 
ject for archival storoge* During the lost three yeors of the plon^ the Heod of the 
Reference Services Unit should devote 5 percent time to the development of de- 
scriptive brochures and other moteriols which inform reseorchers about the 
availability of mochine-reodoble records at the State Archives* 



43 



35 



Summary of SARA Resource Requirements 



Records Analysis and Dispositicii 


FY 1988-89 


FY 1989-90 


FY 1990-91 


FY 1991-92 


FY 1992-93 


Personal Services (FI E) 


170 


1.60 


2 35 






SubTotal-Records Analysis 
and Disposition 


$62,586 


$62,635 


$94,250 


$9L,150 


$99,800 


Agency Services 












Personal Services (FTE) 


••lU 






.55 


.55 


Total Cost (PS) 


$19,500 


$26,000 


$22,000 


$23,100 


$24,250 


NonPersonal Services 


$5,000 


$5,800 


$5,000 


$5,000 


$5,000 


SubTolal-Agency Services 


$24,500 


$31,800 


$27,000* 


$28,100 


$29,250 


Archival Services 












Personal Services (FTE) 


1.35 


1.35 


1.05 


1.05 


1.05 


Total Cost (PS) 


$40,850 


$43,420 


$34,420 


$36,220 


$38,135 


NonPersonal Services 


$22,000 


$232,500 


$23,00G 


$23,500 


$24,000 


Subtotal- Archival Services 


$62,850 


$275,920 


$57,420 


$59,720 


$62,135 


TOTAL COST 


$149,938 


$370,355 


$178,670 


$182,970 


$19L185 



• Does not include possible capital improvements to the Slate Records Center or start-up costs of a possible tape maintenance 
program. 
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